feat: support expansion in gator verify #3650

David-Jaeyoon-Lee · 2024-10-16T21:17:16Z

What this PR does / why we need it:
It allows for expansion in gator verify.

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #3432

Special notes for your reviewer:

codecov-commenter · 2024-10-16T21:27:59Z

Codecov Report

Attention: Patch coverage is 50.74627% with 33 lines in your changes missing coverage. Please review.

Project coverage is 47.72%. Comparing base (3350319) to head (996608e).
Report is 192 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/gator/verify/runner.go	59.64%	16 Missing and 7 partials ⚠️
pkg/gator/reader/read_resources.go	0.00%	10 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (996608e). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (3350319) HEAD (996608e)

unittests 2 1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3650      +/-   ##
==========================================
- Coverage   54.49%   47.72%   -6.78%     
==========================================
  Files         134      236     +102     
  Lines       12329    19846    +7517     
==========================================
+ Hits         6719     9471    +2752     
- Misses       5116     9485    +4369     
- Partials      494      890     +396

Flag	Coverage Δ
unittests	`47.72% <50.74%> (-6.78%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

pkg/gator/errors.go

pkg/gator/verify/runner.go

JaydipGabani · 2024-10-22T21:37:03Z

Are we going to add expannsionTemplate to suite?

maxsmythe

couple small nits

pkg/gator/errors.go

pkg/gator/reader/read_constraints.go

pkg/gator/verify/runner.go

JaydipGabani

@David-Jaeyoon-Lee LGTM after gator tests are modified for expansion template, this might help adding those tests David-Jaeyoon-Lee#2.

David-Jaeyoon-Lee · 2024-10-31T21:08:34Z

@David-Jaeyoon-Lee LGTM after gator tests are modified for expansion template, this might help adding those tests David-Jaeyoon-Lee#2.

I think I added similar tests in this pr already. Or am I misunderstanding something? What specifically do we want to modify?

Edit: Nevermind I see

pkg/gator/errors.go

JaydipGabani · 2024-11-06T20:36:53Z

@David-Jaeyoon-Lee you will probably need to update docs as well to let users know how to use this. This is probably the best place to include that information - https://open-policy-agent.github.io/gatekeeper/website/docs/gator#the-gator-verify-subcommand.

I am ok with a follow up PR to update the docs. @maxsmythe @ritazh @sozercan wdyt?

David-Jaeyoon-Lee · 2024-11-06T22:33:06Z

website/docs/gator.md

@@ -128,7 +128,7 @@ gator test --filename=manifests-and-policies/ --output=json
 `gator verify` organizes tests into three levels: Suites, Tests, and Cases:

 - A Suite is a file which defines Tests.
- A Test declares a ConstraintTemplate, a Constraint, and Cases to test the
+- A Test declares a ConstraintTemplate, a Constraint, an ExpansionTemplate (optional), and Cases to test the


Is it necessary to add here or should it just be a separate section.

David-Jaeyoon-Lee · 2024-11-06T22:33:07Z

website/docs/gator.md

@@ -162,6 +162,8 @@ ConstraintTemplate. It is an error for the Constraint to have a different type
 than that defined in the ConstraintTemplate spec.crd.spec.names.kind, or for the
 ConstraintTemplate to not compile.

+A Test can also optionally compile an ExpansionTemplate.


David-Jaeyoon-Lee · 2024-11-06T22:33:09Z

website/docs/gator.md

@@ -264,6 +266,25 @@ the `run` flag:
 gator verify path/to/suites/... --run "disallowed"
 ```

+### Validating Generated Resources with ExpansionTemplates


Not sure if this is too much or if its not enough (i.e. no example or a more complete example).

JaydipGabani

@David-Jaeyoon-Lee Thanks for the doc changes. LGTM

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

…or gator verify Signed-off-by: David-Jaeyoon-Lee <[email protected]>

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

sozercan

Thanks! LGTM

maxsmythe

LGTM

malexander2012 · 2024-11-22T21:42:24Z

Thank you to all who were involved in making this happen!

David-Jaeyoon-Lee requested a review from a team as a code owner October 16, 2024 21:17

David-Jaeyoon-Lee marked this pull request as draft October 16, 2024 21:17

JaydipGabani reviewed Oct 22, 2024

View reviewed changes

pkg/gator/errors.go Outdated Show resolved Hide resolved

pkg/gator/verify/runner.go Outdated Show resolved Hide resolved

David-Jaeyoon-Lee marked this pull request as ready for review October 24, 2024 22:55

David-Jaeyoon-Lee requested a review from JaydipGabani October 25, 2024 20:55

maxsmythe requested a review from a team October 25, 2024 23:52

maxsmythe requested changes Oct 25, 2024

View reviewed changes

pkg/gator/errors.go Outdated Show resolved Hide resolved

pkg/gator/reader/read_constraints.go Outdated Show resolved Hide resolved

pkg/gator/verify/runner.go Show resolved Hide resolved

ritazh reviewed Oct 29, 2024

View reviewed changes

pkg/gator/verify/runner.go Show resolved Hide resolved

ritazh added this to the v3.18.0 milestone Oct 30, 2024

David-Jaeyoon-Lee commented Oct 30, 2024

View reviewed changes

pkg/gator/verify/runner.go Show resolved Hide resolved

David-Jaeyoon-Lee requested review from maxsmythe and ritazh October 30, 2024 19:59

David-Jaeyoon-Lee force-pushed the davjlee/feature/gator-verify-expansion branch from b15146f to f9e6699 Compare October 30, 2024 20:16

JaydipGabani requested changes Oct 31, 2024

View reviewed changes

David-Jaeyoon-Lee requested a review from JaydipGabani November 4, 2024 19:59

David-Jaeyoon-Lee force-pushed the davjlee/feature/gator-verify-expansion branch from d3892b4 to 8339ab4 Compare November 4, 2024 21:25

JaydipGabani approved these changes Nov 4, 2024

View reviewed changes

ritazh requested changes Nov 6, 2024

View reviewed changes

pkg/gator/errors.go Outdated Show resolved Hide resolved

pkg/gator/errors.go Outdated Show resolved Hide resolved

David-Jaeyoon-Lee requested a review from ritazh November 6, 2024 04:41

ritazh approved these changes Nov 6, 2024

View reviewed changes

David-Jaeyoon-Lee commented Nov 6, 2024

View reviewed changes

David-Jaeyoon-Lee requested review from ritazh and JaydipGabani November 6, 2024 22:33

JaydipGabani approved these changes Nov 6, 2024

View reviewed changes

David-Jaeyoon-Lee added 8 commits November 13, 2024 22:09

Add expansion template support for gator verify

9930bdf

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

fix error handling

9aa60fb

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

fix lint errors, error handling, and unused functions

4c16047

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

add testing for gator verify expansion template support

1215c23

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

Rebase to master & fix merge conflicts

be6e9f6

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

Add/update resources for testing expansion under the test directory f…

cb5cab8

…or gator verify Signed-off-by: David-Jaeyoon-Lee <[email protected]>

Fix comments on errors.go

104fafc

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

Add gator verify expansion support details in website docs

6ac1628

Signed-off-by: David-Jaeyoon-Lee <[email protected]>

David-Jaeyoon-Lee force-pushed the davjlee/feature/gator-verify-expansion branch from 6c1464a to 6ac1628 Compare November 13, 2024 22:10

sozercan approved these changes Nov 13, 2024

View reviewed changes

Merge branch 'master' into davjlee/feature/gator-verify-expansion

9103f81

maxsmythe approved these changes Nov 14, 2024

View reviewed changes

Merge branch 'master' into davjlee/feature/gator-verify-expansion

996608e

ritazh approved these changes Nov 14, 2024

View reviewed changes

ritazh merged commit 31932a2 into open-policy-agent:master Nov 14, 2024
20 checks passed

BrewTestBot mentioned this pull request Dec 12, 2024

gator 3.18.0 Homebrew/homebrew-core#200877

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support expansion in gator verify #3650

feat: support expansion in gator verify #3650

David-Jaeyoon-Lee commented Oct 16, 2024

codecov-commenter commented Oct 16, 2024 •

edited

Loading

JaydipGabani commented Oct 22, 2024

maxsmythe left a comment

JaydipGabani left a comment

David-Jaeyoon-Lee commented Oct 31, 2024 •

edited

Loading

JaydipGabani commented Nov 6, 2024

David-Jaeyoon-Lee Nov 6, 2024

David-Jaeyoon-Lee Nov 6, 2024

David-Jaeyoon-Lee Nov 6, 2024

JaydipGabani left a comment

sozercan left a comment

maxsmythe left a comment

malexander2012 commented Nov 22, 2024

feat: support expansion in gator verify #3650

feat: support expansion in gator verify #3650

Conversation

David-Jaeyoon-Lee commented Oct 16, 2024

codecov-commenter commented Oct 16, 2024 • edited Loading

Codecov Report

JaydipGabani commented Oct 22, 2024

maxsmythe left a comment

Choose a reason for hiding this comment

JaydipGabani left a comment

Choose a reason for hiding this comment

David-Jaeyoon-Lee commented Oct 31, 2024 • edited Loading

JaydipGabani commented Nov 6, 2024

David-Jaeyoon-Lee Nov 6, 2024

Choose a reason for hiding this comment

David-Jaeyoon-Lee Nov 6, 2024

Choose a reason for hiding this comment

David-Jaeyoon-Lee Nov 6, 2024

Choose a reason for hiding this comment

JaydipGabani left a comment

Choose a reason for hiding this comment

sozercan left a comment

Choose a reason for hiding this comment

maxsmythe left a comment

Choose a reason for hiding this comment

malexander2012 commented Nov 22, 2024

codecov-commenter commented Oct 16, 2024 •

edited

Loading

David-Jaeyoon-Lee commented Oct 31, 2024 •

edited

Loading